Using mutual information to measure order in model glass-formers 
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Whether or not there is growing static order accompanying the dynamical heterogeneity and 
increasing relaxation times seen in glassy systems is a matter of dispute. An obstacle to resolving 
this issue is that the order is expected to be amorphous and so not amenable to simple order 
parameters. We use mutual information to provide a general measurement of order that is sensitive 
to multi-particle correlations. We apply this to two glass-forming systems (2D binary mixtures of 
hard disks with different size ratios to give varying amounts of hexatic order) and show that there is 
little growth of amorphous order in the system without crystalline order. In both cases we measure 
the dynamical length with a four-point correlation function and find that it increases significantly 
faster than the static lengths in the system as density is increased. We further show that we can 
recover the known scaling of the dynamic correlation length in a kinetically constrained model, the 
2-TLG. 

PACS numbers: 02.50.Ey 89.70,+c 89.75.Kd 05.20.Jj 64.70.kj 



I. INTRODUCTION 

A central question in the physics of glass forming liquids 
is whether or not they develop static structure when su- 
percooled (or compressed), and whether such structure 
plays an important role in the extreme slowing down that 
occurs as a system approaches the glass transition. Of- 
ten, we expect the increase of relaxation time in a system 
to be accompanied by an increasing length-scale [JJG] and 
this should be the case if glassy phenomena are related to 
some sort of critical behaviour [2]. It is well established 
that many supercooled liquids exhibit a growing dynamic 
length-scale [3] but pairwise correlation functions show 
little change in the structure upon cooling 0]. 

A number of theoretical scenarios have been postulated 
to explain glassy behaviour. Some include static struc- 
ture: such as clusters of locally- favoured order [5j [6] ; or 
a mosaic of finite regions of amorphous order [7] . There 
are alternative explanations based on dynamic facilita- 
tion effects [5] or a dynamical phase transition [5]. Nei- 
ther of these scenarious require static structure. Futher- 
more, it is possible to produce glass-like behaviour with 
kinetically constrained models [JU] E] that are designed 
to omit static structure, although these are not derived 
from the microscopic behaviour of actual glasses. 

So, the necessity of a growing static length-scale is ques- 
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tionable. There is good evidence that some change in 
structure (not necessarily related to a growing length 
scale) occurs on dynamic slowing |12H16j . Some nu- 
merical studies indicate a growing static length-scale 
[T1HT7] and there are thermodynamic treatments [TH] 
which imply an increasing length-scale (although it is 
not measured directly from the real space configura- 
tion of particles). Other experimental [TH [20] and 
numerical [2TJ work suggests that this length-scale in- 
creases only slightly. There is disagreement over whether 
static lengths in glassy systems grow with the dynamical 
length-scale or not [IB ] 1X7 ] I2TH23] . 

To determine exactly what is going on we need a tech- 
nique that can measure amorphous order directly in a 
given system. In some systems there is a clear idea of 
what this order should look like and it can be measured 
with, for example, a bond-orientational order parameter 
[JJJ [J5] . The problem in these cases is that the technique 
is not general and systems with such clear and simple or- 
dering may not be characteristic of glassy systems. A less 
specific approach is to look for geometrical motifs [JS] al- 
though this still requires that we limit our search to a set 
of predefined structures. Generalised order parameters 
have been suggested: local "structural entropy" s 2 [TB] 
does not rely on presupposed order but it may be con- 
founded by dynamical information and it is only sensitive 
to pairwise correlations; the configurational entropy ap- 
proach to measuring patch-correlation length [2H HSJ is 
certainly general although it is computationally unfeasi- 
ble for the system studied here. We discuss both of these 
later in the text. 
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It would be useful to have a general method of measur- 
ing structure that does not require us to specify in ad- 
vance what we are looking for and that is not blind to 
things that we did not expect. Information theory gives 
us a framework in which we can look for structure in an 
order-agnostic way. Using the concept of mutual infor- 
mation (see e.g. |26j ) we can quantify all of the depen- 
dencies between two multi-dimensional random variables. 
This enables us to develop techniques that are sensitive 
to higher order correlations and that do not depend on 
the structure in the system taking a presumed form. 

We use the mutual information between patches in a sys- 
tem's configuration as a general measurement of order. 
We measure this quantity in a model glass-forming sys- 
tem: a 2D binary mixture of hard-disks. The size ratio of 
the disk species is varied to alter the amount of hexatic 
order in the system. We derive a static length-scale from 
these mutual information measurements: it grows in tan- 
dem with the hexatic order correlation length in the hex- 
atic system as density is increased. The mutual informa- 
tion length in the non-hexatically-ordered system varies 
little as density is changed. In both cases the growth 
in the dynamical length-scale significantly exceeds that 
of the static lengths. We also apply these methods to a 
kinetically-constrained model. 

The paper is structured as follows. We briefly review 
the concept of mutual information and discuss how it 
can be used to look for static and dynamic structure in 
super-cooled liquids. We investigate two systems which 
exhibit glassy behaviour: a 2D off-lattice binary hard 
disk mixture and a kinetically constrained model, the 
2-TLG (lattice-based, and also 2D). We describe the dis- 
cretisation procedure we use to obtain a symbolised rep- 
resentation of patches in the system's configuration and 
then calculate the mutual information between pairs of 
patches. These measurements are used to define a mu- 
tual information length of static order in the system. We 
investigate the behaviour of this length as the density of 
the system is varied, and compare the lengths to the dy- 
namic correlation length in the system. The similarities 
between this approach and the patch-correlation length 
of [25] will be discussed. 



II. INFORMATION THEORY 

Here we discuss the application of information theory to 
extract a structural or dynamic length-scale. The fun- 
damental information theoretic quantity is the Shannon 
entropy [37] . The Shannon entropy of a random variable 
X with a probability distribution p(x) over a support X 
is given by: 

H(X) = -^2p(x)log 2 p(x) (1) 

xex 



This quantity is larger for a uniform probability distri- 
bution over a broad support (phase space), and smaller 
when the support gets smaller, or the distribution more 
peaked. It is a measure of the uncertainty of the outcome 
of drawing a sample from the distribution. 

When measuring structure, we are interested in how the 
configuration in one part of a system affects the configu- 
ration in another. We can think about this in information 
theoretic terms. If the configuration, X, in some part of 
the system (we call this a patch) influences that in an- 
other part, Y, then it will be the case that when X is 
held constant the range of possible values of Y is smaller 
than when X can take any value. We can quantify this 
reduction in uncertainty by treating our configurations 
as random variables and taking the mutual information 
(see e.g. [26]). 

The mutual information between two random variables 
measures the entropy difference between the marginal 
probability distribution of a variable, and its conditional 
distribution. 

I(X; Y) = H(X) - H{X\Y) = H(Y) - H(Y\X) (2) 
= H(X) + H(Y) - H(X,Y) (3) 

= £ p ( Xl y)lo g2 ^f- (4) 

The mutual information can be thought of as a distance 
(although not rigorously) between the true joint distri- 
bution of the two variables, and the distribution they 
would have if they were independent. In terms of config- 
urations, the mutual information will be zero when X has 
no influence on7; it will be positive and increasing as X 
becomes more influential; and it will take its maximum 
value I(X; Y) = H(Y) if Y is completely determined by 
X. The mutual information is symmetric in X and Y . 

We have a choice in the shape of our patches, X and Y. 
When measuring mutual information in time-series data 
it is intuitive to divide the system at a nominal present 
time, t, and measure the mutual information between the 
output over some past period (t — t — > t) and the future 
output (from t — > t + t) 28 . By varying r it is possible 
to measure not only the amount of information the past 
of the system holds about the future, but also the length 
of time information persists in the system. 

Here we are looking at spatial data: an analogous ap- 
proach would be to divide the system in two and measure 
mutual information between configurations either side of 
the divide. However, this gives configuration spaces that 
are too large to sample. It is possible to approximate this 
approach (perfectly, under certain conditions) by measur- 
ing the mutual information of two abutting patches and 
varying their length (in the direction away from their in- 
terface) [29]. The configuration space is smaller, but pro- 
portional to the length of the patches. Therefore, it can 
still be too big when the patches are made long enough 
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to encompass long-range correlations. A computationally 
cheaper method is to measure the mutual information be- 
tween two patches that are not abutting. Correlations at 
different lengths can be measured by varying the separa- 
tion of the patches rather than increasing their size. 

The patch correlation length of [25] is based on the en- 
tropy of single patches rather than the mutual informa- 
tion between patches. The patches are centred on parti- 
cles so each patch represents the configuration of particles 
within a radius, r of a given particle. Two patches are 
said to belong to the same state if these configurations 
are the same (some difference is allowed for thermal vi- 
bration). The configurational entropy is calculated by 
comparing all of the particles in the system and taking 
the entropy of the distribution of states. This approach 
was inspired by the random first-order transition theory 
of glasses [7] which considers the system a mosaic of or- 
dered tiles. This order ensures that the entropy increases 
sub-extensively with r until patches start to encompass 
multiple tiles. As r is increased beyond this the entropy 
becomes extensive: the patch correlation length measures 
this crossover. 

Alternatively, one could measure the mutual information 
between two separate patches in the system. If these 
are within a single tile the mutual information will be 
positive; if they do not share any tiles then the mutual 
information will be zero. If there is a crossover from 
sub-extensive to extensive regimes in the configurational 
entropy then this will be represented in the mutual in- 
formation. 

The advantage of our mutual information approach is 
that we have defined our patches in such a way to probe 
different distances without increasing the size of the 
patches. As patch size increases it becomes harder to 
sample the patch distribution well: the method used in 
[25] can measure a maximum entropy of log N. N is the 
number of particles (hence patches) in the system so the 
maximum is reached when all N patches are in unique 
states. In fact, the entropy should be lower than this 
maximum to ensure that all possible states have had a 
chance to be sampled. We reached this limit for small 
patches when measuring the configurational entropy for 
the binary hard-disk system whereas we were able to ap- 
ply our method for measuring the mutual information 
between patches successfully. Our method does discard 
some information: it will not account for multipoint cor- 
relations that involve particles in the gap between the 
two patches; however, it will still detect the length over 
which static order extends in the system. 



III. METHODS 

Simulation details. We investigate the mutual informa- 
tion in computer simulations of two systems. The first 



is a binary mixture (50:50) of small (radius a) and large 
(Rcr) hard disk particles in 2D. We look at systems with 
R = 1.4 and R — 1.2. In both cases the system ex- 
hibits dynamic slowing down (Fig. [3]) and other glassy 
behaviour (e.g. dynamic heterogeneity, Figs. [5] and EJ), 
but at R = 1.2 there is much more crystalline order. We 
look at systems with area fractions 0.70 < <fr < 0.80. 
Monodisperse hard disks undergo a liquid-hexatic tran- 
sition at around </> = 0.71 and a hexatic-solid transition 
at around = 0.72 [30] . 

The system evolves with Monte Carlo dynamics: a trial 
move involves shifting a particle to a random position 
somewhere in an 0.05 x 0.05a 2 square centred on its orig- 
inal position. If the move does not lead to an overlap with 
any other particle then the move is accepted. We mea- 
sure time in Monte Carlo sweeps: one sweep involves N 
attempted moves, where N is the number of particles in 
the system (N — 20000). Periodic boundary conditions 
are used. 

The second system is the two-vacancy-assistcd-hopping 
triangular lattice gas (2-TLG). This is a lattice gas model 
of a glass-forming fluid introduced in |llj . Hard parti- 
cles sit on a two-dimensional triangular lattice. Monte 
Carlo dynamics are used to evolve the system: a random 
particle is chosen and an attempt made to move it to a 
random neighbouring site. The move is only accepted if 
the neighbouring site is vacant, and if both sites that are 
mutual neighbours of the particle's starting site and the 
trial-move site are also vacant (see Fig. [I]). As the system 
is a lattice gas there is no static structure. However, the 
(2)-TLG is known to slow down dramatically (see Fig. [9]) 
and become increasingly heterogeneous as its density is 
increased [TO] . 




FIG. 1: The kinetic constraints in the 2-TLG model: for 
the central particle to move into the right-most vacancy it 
passes through the mutually neighbouring sites. These must 
be empty for the move to be accepted. 

Mutual information based measurements of structure. In 
the binary disk system we take 1.8 x 1.8a 2 square patches 
(each patch is large enough to hold up to six small par- 
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FIG. 2: The system for discretising the patch configuration 

in the MC hard disk system (not to scale). A grid is 
superimposed on the patch and pixels that include a particle 
centre are marked 1 or 2 depending on particle type. Empty 
space is encoded with zero. The patches are represented by 
a 3-ary number. Mutual information is measured between 
two patches separated by a distance da. 



tides). The distance between the patch centres is d as 
shown in Fig. [2j We discretise the patches onto a 6 x 6 
grid and represent each configuration as a 3-ary number 
(0 - empty space; 1 - small particle; 2 - large particle). We 
assume the joint probability distribution of two patches 
separated by a given distance (note that the displacement 
is always along the axis of the square) is stationary over 
space. As such, we sample the probability distribution 
over space and multiple instances of the system: the po- 
sitions and orientations of the sampled pairs of patches 
are chosen at random. 

For the TLG there is no need to discretise the patches. 
We use hexagonal patches (of varying side length) and 
encode them as binary numbers (as there is only one 
particle type). 

Mutual information is calculated directly from the his- 
togram of patch values. It is possible to calculate mutual 
information from continuous distributions |31j but the 
nature of the patches - they contain an unknown number 
of either type of particle and therefore do not have a set 
dimensionality - makes this difficult. These issues do not 
affect the discretised patches, which always consist of n 2 
symbols. By the information-processing inequality [26] 
the mutual information between discretised patches is a 
lower-bound on the mutual information between contin- 
uous patches. 

The histogram method for estimating mutual informa- 
tion has a positive systematic bias. Finite-sample correc- 
tions exist [35] [US] although we obtained better results 
by measuring entropies at different sample numbers and 
fitting curves to estimate errors (see appendix [A]) . Equa- 
tion [3] is used to calculate mutual information from the 
entropy measurements. The mutual information static 
length (£ m i) is defined as the first moment of the dis- 
tribution of patch mutual information with separation, 
d. 



Y, d i{X;Y d ) 



(5) 



When d < 1.8a some of the mutual information mea- 
sured is due to the overlap between the two patches. To 
avoid this we measure the mutual information between 
one full-size patch and the non-overlapping part of the 
other patch. We normalise the mutual information by its 
theoretical maximum (the entropy of the larger patch) to 
give a value that is comparable at all d. 

Conventional measurements of structure. We wish to 
compare any static structure we might find to the length- 
scale of the dynamical heterogeneity in the system. To 
do so, we calculate the dynamic length using a four- 
point correlation function approach similar to that in |34j 
which is described in appendix [B| 

We also compare the mutual information lengths to other 
order parameters in the binary hard disk systems: the 
hexatic order parameter 4^ = Ylken(j) ex P[^jk] ( n (j) 
are the neighbours of particle j; 9jk is the angle between 
particles j and fc) [35] ; and local S2 [Ej . In both cases a 
length-scale is extracted by fitting a 2D Ornstein-Zernike 
envelope [16] to the normalised correlation function of 
the order parameter (see Fig. [7]). g x (r) is the correlation 
function and £ x the length-scale of a particular order pa- 
rameter: here we use g§{r) and g s ^{r) for the correlation 
functions of 'J 6 and local s 2 respectively. 



9x{r) 

g( r ) 



oc r 1/4 exp(-r/^ x ) 



(6) 



Local s 2 is calculated from the individual pair correlation 
functions gf(r) and g\{r) 



9f(r) 



9\{r) 
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(7) 



(8) 
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A(B) is the set of small (large) particles. N is the number 
of particles in the system and p is the system density. 

We average over 10t q to remove short-term fluctuations 
and to ensure that g{r) is adequately sampled. The fi- 
nal particular s 2 is the sum of contributions from both 
correlation functions: 



P 
2 



fc£{a,6} 



dr 



fc*(r)_fl?(r)-(fl?(r)-l) (9) 



The integration of s 2 should ideally be between zero and 
+oo. Practically, it is cut off at a value that is large 
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enough to take in many shells of surrounding particles. 
This may be slightly different to the method implemented 
in [IB] but we believe out method should capture any 
static length-scale. 




FIG. 3: The relaxation time r a against area fraction <f> for 
the two values of 7? in the hard disk system. r a is measured 
as the time taken by the self-intermediate scattering 
function to fall to exp(-l) [SB]. The lines are VFT fits: 
r a oc exp{A/(<j>o - </>))• 4>o equals 0.82 (0.83) for the R = 1.2 
(1.4) systems. 



IV. RESULTS 



A. Binary hard disks 



As <t> is increased from 0.7 the relaxation times in both 
systems increase significantly (Fig. [3]) with the R = 1.2 
system slowing more. The relaxation times, r a , are mea- 
sured from self intermediate scattering functions. Vogel- 
Fulcher-Tamman fits (r Q = r exp[D^>/(0o — </>)]) are used 
to obtain fa, the ideal glass transition packing fraction. 
This is used later when comparing length-scales. 

Both systems also become dynamically heterogeneous. 
Plots of Xi (Fig- |4| indicate that both systems are 
dynamically heterogeneous over intermediate times and 
that the maximum heterogeneity increases as the system 
becomes more dense. Figure [5] shows the dynamical cor- 
relation lengths calculated at the time of maximum Xi 
for each <j>. These show the range of the dynamic correla- 
tions increasing with 4>. Again, this is more pronounced 
in the R = 1.2 system. The details of how Xi an d £4 were 
calculated are in appendix [B| 

We begin by considering the distribution of mutual in- 
formation with patch separation distance (Fig. [6]). There 
is little change in the distribution with <f> for the R = 1.4 
system: the mutual information increases slightly at 
short distances and there is a small but consistent in- 



crease in the mutual information length as the density is 
increased. This is in marked contrast to the R = 1.2 sys- 
tem (as is be expected given its hexatic ordering). In this 
case the mutual information decays much more slowly 
with distance at the higher density state points. The 
increase in mutual information length seems consistent 
with the increase in £ 6 - 

This is apparent in Fig. [8] where the various lengths are 
plotted against the reduced area fraction, 0o — </>■ The 
mutual information and \1/ 6 lengths (£ m ; and £g) increase 
at similar rates in the R — 1.2 system. However neither 
increases as fast as the dynamical heterogeneity length, 
£4. The difference in static and dynamic length-scales 
is more pronounced in the R = 1.4 system. Here the 
mutual information length barely increases at all whereas 
the dynamic length at (f> = 0.8 is an order of magnitude 
greater than at <f> = 0.7. 

As the hexatic order parameter varies little for the R = 
1.4 system we measure the local s 2 length. The corre- 
lations in this quantity have been used to detect order 
in various glass- forming liquids |16j . It measures (in its 
global form) the pairwise contribution to extra configu- 
rational entropy of the system compared to an ideal gas 
|37j . Unlike the static mutual information length, £ s 2 
increases with system density (although not as much as 

U). 

Essentially, s 2 measures the peakedness of a pair correla- 
tion function. It is averaged, in this case, over a trajec- 
tory of the system: so a particle that moves little and is 
surrounded by similar particles will have a rather spiked 
g(r); particles that move around a lot will smooth out 
their g(r). The peakedness of individual g(r) that are 
averaged in this way will be sensitive to dynamic het- 
erogeneity. If we assume mutual information measures 
any true increase in structural correlation length, it is 
uncertain whether increasing £ s 2 is measuring an increas- 
ing static length or is confounded by the increasing dy- 
namic length-scale. The fact that the mutual information 
length shows no such increase suggests that the second 
possibility is likely. 

Finally, we look at the dynamic mutual information be- 
tween patches. This is calculated similarly to the static 
mutual information, but we no longer encode the parti- 
cle type and instead encode the mobility of the particle. 
The mobility is taken from the overlap function (Equa- 
tion Bl ) used to calculate the four-point correlation func- 



tions. The patch is constructed such that mobile parti- 
cles are encoded with a one and immobile particles and 
empty space are encoded with zero. The information the 
patches contain is the position of mobile particles in the 
system. Unlike the static mutual information, the dy- 
namic mutual information shows a notable increase at 
short distances and extends to longer distances as <j> is 
increased (Fig. [6]). 
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FIG. 4: Xiif) f° r the binary hard disk system at R = 1.2 and R = 1.4 . The time of the maximum Xi(^ — T h) is used when 

calculating the length, £4. 
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original position after time t then it is mobile. 

Figure [To] shows the dynamic mutual information length, 
£dyn,mi(i), measured at different t for various system 
densities (p). It is clear that dynamical heterogeneity 
is an intermediate time-scale phenomenon: each curve 
peaks at a time proportional to the relevant relaxation 
time (t q ). As expected, this maximum length increases 
as the system becomes more dense. Figure 10 also 
shows the dynamic mutual information lengths at r Q , 
?dyn,mi( T a): compared to existing four-point correlation 
measurements (from |10j). The scaling of length with re- 
laxation time with exponent 1.4 agrees well for both sets 
of measurements. 



FIG. 5: The dynamic correlation length £4 increases with 

area fraction for both R — 1.2 and R — 1.4. As with the 
increase in relaxation time, the effect is greater for R = 1.2. 
Lines are a guide to the eye. 



B. Triangular Lattice Gas 



The 2-TLG, being a kinetically constrained model, has 
no static structure by design. As expected, no static 
structure was found using patch mutual information. We 
calculated the dynamic mutual information (as described 
above) using patches of various radii. Here we focus on 
the lengths obtained from radius one (i.e. point) patches. 
These contain no extra structural information and so 
are directly comparable to the dynamic lengths obtained 
from four-point correlation functions. The dynamic mu- 
tual information lengths are obtained as before (Equa- 
tion [5| . The mobility of particles is again measured by 
an overlap function: if the particle has moved from its 



DISCUSSION AND CONCLUSIONS 



In this paper, we have shown that the mutual informa- 
tion between patches can be used to measure static order 
in amorphous systems. It is possible to extract a length- 
scale from such measurements, and in the system where 
we had direct access to a relevant order parameter the 
mutual information length was in agreement. The dy- 
namic mutual information measurements are useful in 
that they show the difference in the growth of correla- 
tions in the R = 1.4 hard disk system as <j) increases. 
However, it is not an optimal way to measure dynamic 
heterogeneity: whereas the 4-point-correlation function 
[54] depends only on the mobility of pairs of particles at 
some distance, the dynamic mutual information contains 
unneeded information of the configuration of mobile par- 
ticles within the patches. In the case where the p atches 
are reduced to point size (as for the 2-TLG, Section IV B ) 



this issue is removed and we recover the behaviour of the 
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(b) R=1.4 
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FIG. 6: Plots of the mutual information between patches {1(d)) against patch separation d for the binary hard disk system 
with R = 1.2 and R — 1.4. The values have been corrected for overlapping patches and normalised so that at each distance 

the maximum possible mutual information is one. The left and central plots are of mutual information between patches 
encoding static information. The right plot shows mutual infomration betwen patches encoding dynamic information for the 
R — 1.4 system. The (static) mutual information lengths (and the ^t 6 length for R — 1.2) are plotted in the insets. 
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FIG. 7: Ornstein-Zernike fits and lengths of ty G for R = 1.2 (a,c) and R — 1.4 (b,d). The lower plots show the extracted 
lengths (£e) against <j>. There is a clear increase in the length of hexatic order for the R = 1.2 system (c) that is absent when 

R= 1.4(d). 




FIG. 8: A comparison of lengths in the binary hard disk R = 1.2 and R = 1.4 systems. The lines are fits of £ oc (4>a ~ 4>) b 
where <f>o is taken from the VFT fit of r a for the system. £4 is the dynamical correlation length; £ m j is the (static) mutual 
information length; and ^6 and £ s 2 are the correlation lengths of ^ 6 and local s 2 respectively. The exponent for £ m i in the 
R — 1.4 system is b = —0.1 with a standard error of 0.01: as density is increased the growth in £ m i is small, but non-zero. 
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FIG. 9: Relaxation time against density for the 2-TLG. 



be noted that we did have trouble measuring s 2 exactly 
as specified in 16j and so this may have influenced our 
results. 

These conclusions are compatible with dynamical facili- 
tation [8] or dynamical phase-transition [38] explanation 
of the glass transition, rather than scenarios which invoke 
strongly growing static order such as Random First Order 
Theory [7|. It may be relevant that the hard disk sys- 
tem investigated here has purely repulsive interactions. 
There is reason to suppose that local structure is more 
important in systems with attractive potentials |39j . It 
is possible that the mutual information between patches 
would behave differently in such systems. 
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Appendix A: Compensating for finite sample errors 

We estimate each probability distribution with the fre- 
quency distribution obtained by sampling. As we have 
only a finite number of samples we may not encounter 
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FIG. 10: (a) the dynamic mutual information length 
(£dyn,mi(i)) for the 2-TLG at different densities and with 
mobility measured over different times, t; (b) the dynamic 

mutual information length at t = r a for each density 
(£dyn,mi(T"a) - red triangles). The blue circles are dynamic 
heterogeneity length data from [10] for the 2-TLG. For 
higher densities (higher r a ) the lengths scale similarly. The 
inset shows ^dyn^m^a) plotted against density. 



some low probability patch configurations and therefore 
our estimate of the support of the probability distribu- 
tion will be too small. Also, the frequencies we measure 
will fluctuate from their true values which will have the 
effect of making the estimated distribution less uniform 
than the true distribution. Both of these effects cause 
a systematic underestimation in entropy. The effect in- 
creases with the size of the probability space (holding 
the number of samples constant) and so when estimating 
mutual information using Equation [3j it is the negative 
H(X,Y) term that dominates the error. As such, the 
mutual information will have a positive systematic bias. 
Figure [XT] shows this effect: there is positive mutual in- 
formation at long distances when we would expect none. 
This effect reduces as the sample size is increased. 

To estimate the true entropy, lim^oo H(X, Yd) for two 
patches separated by a distance d we measure the en- 
tropy at various sample sizes: H n (X, Yd) where n is pro- 



portional to the number of samples. We assume that the 
difference between the true and finite sample entropies is 
given by a series of terms [52] : 



By fitting our data to this form we can estimate H^. 

Figure [TT] shows such a fit using only the first order error 
term (this technique was used for the R = 1.4 systems). 
In this case 6 = 0.5 gives a good fit for all non-overlapping 
patches. The exact value of b varies between the systems 



and decreases as H increases (Fig. 12). 



By ignoring higher order terms we overestimate the error 
and get (unphysical) negative mutual information values. 
To compensate for this we shift the curve so that the 
baseline is zero before measuring the mutual information 
length. Adding the second order term decreases this error 
although it makes no significant difference to the mutual 
information length. 
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FIG. 11: The grey lines in the main plot are mutual 
information estimates (without overlap corrections) with 
different sample sizes: the number of samples doubles each 
time from top to bottom. The red line is the estimated true 
entropy, Hoo^X, Yd). The inset shows the first-order error fits 
that were used to obtain H ao (X, Yd) for various values of d. 



Appendix B: Dynamic correlation measurements 



We derive the dynamic correlation length using a four- 
point correlation function approach similar to that in 
|34) . To start with we calculate an overlap function for 
each particle. 



Wi(t) 



1 if|r t (<)-r 4 
otherwise 
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FIG. 12: The exponent in the error terms versus the 
estimated true entropy. The different coloured points 
represent systems with different <j>. Triangles are R = 1.2 
systems; circles are R = 1.4 systems 



To find the time span, r^, over which the system is most 
dynamically heterogeneous we calculate 



X4(t) 



l r 



(B2) 



Where Q(t) = J2i w i(t)- The averages are taken over 
many realisations of the system. For each density we 
find 7V the time which maximises Xi(t) an d use it to 
calculate a structure factor: 

Si(k) = '^(wj(Th)-w(Th))(wj(Th)-w(Th)) exp (ifc Ary, ) 

ij 

(B3) 

(k is spatial frequency). This is circularly averaged and 
an Ornstein-Zernike function is fit to the low k part of 
our data to give £4, the dynamic length: 



S 4 (k) 



So 



1 + (fc£ 4 ) 2 



(B4) 



Figures |4] and [5] show % 4 and £4 against <j>. As expected, 
the maximum heterogeneity is greater and occurs later 
for higher density systems. The lengths £4 grow rapidly 
as 6 is increased. 
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